36 research outputs found
Understanding and Comparing Deep Neural Networks for Age and Gender Classification
Recently, deep neural networks have demonstrated excellent performances in
recognizing the age and gender on human face images. However, these models were
applied in a black-box manner with no information provided about which facial
features are actually used for prediction and how these features depend on
image preprocessing, model initialization and architecture choice. We present a
study investigating these different effects.
In detail, our work compares four popular neural network architectures,
studies the effect of pretraining, evaluates the robustness of the considered
alignment preprocessings via cross-method test set swapping and intuitively
visualizes the model's prediction strategies in given preprocessing conditions
using the recent Layer-wise Relevance Propagation (LRP) algorithm. Our
evaluations on the challenging Adience benchmark show that suitable parameter
initialization leads to a holistic perception of the input, compensating
artefactual data representations. With a combination of simple preprocessing
steps, we reach state of the art performance in gender recognition.Comment: 8 pages, 5 figures, 5 tables. Presented at ICCV 2017 Workshop: 7th
IEEE International Workshop on Analysis and Modeling of Faces and Gesture
Unmasking Clever Hans Predictors and Assessing What Machines Really Learn
Current learning machines have successfully solved hard application problems,
reaching high accuracy and displaying seemingly "intelligent" behavior. Here we
apply recent techniques for explaining decisions of state-of-the-art learning
machines and analyze various tasks from computer vision and arcade games. This
showcases a spectrum of problem-solving behaviors ranging from naive and
short-sighted, to well-informed and strategic. We observe that standard
performance evaluation metrics can be oblivious to distinguishing these diverse
problem solving behaviors. Furthermore, we propose our semi-automated Spectral
Relevance Analysis that provides a practically effective way of characterizing
and validating the behavior of nonlinear learning machines. This helps to
assess whether a learned model indeed delivers reliably for the problem that it
was conceived for. Furthermore, our work intends to add a voice of caution to
the ongoing excitement about machine intelligence and pledges to evaluate and
judge some of these recent successes in a more nuanced manner.Comment: Accepted for publication in Nature Communication
Explaining Predictive Uncertainty by Exposing Second-Order Effects
Explainable AI has brought transparency into complex ML blackboxes, enabling,
in particular, to identify which features these models use for their
predictions. So far, the question of explaining predictive uncertainty, i.e.
why a model 'doubts', has been scarcely studied. Our investigation reveals that
predictive uncertainty is dominated by second-order effects, involving single
features or product interactions between them. We contribute a new method for
explaining predictive uncertainty based on these second-order effects.
Computationally, our method reduces to a simple covariance computation over a
collection of first-order explanations. Our method is generally applicable,
allowing for turning common attribution techniques (LRP, Gradient x Input,
etc.) into powerful second-order uncertainty explainers, which we call CovLRP,
CovGI, etc. The accuracy of the explanations our method produces is
demonstrated through systematic quantitative evaluations, and the overall
usefulness of our method is demonstrated via two practical showcases.Comment: 12 pages + supplemen
Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Models
State-of-the-art machine learning models often learn spurious correlations
embedded in the training data. This poses risks when deploying these models for
high-stake decision-making, such as in medical applications like skin cancer
detection. To tackle this problem, we propose Reveal to Revise (R2R), a
framework entailing the entire eXplainable Artificial Intelligence (XAI) life
cycle, enabling practitioners to iteratively identify, mitigate, and
(re-)evaluate spurious model behavior with a minimal amount of human
interaction. In the first step (1), R2R reveals model weaknesses by finding
outliers in attributions or through inspection of latent concepts learned by
the model. Secondly (2), the responsible artifacts are detected and spatially
localized in the input data, which is then leveraged to (3) revise the model
behavior. Concretely, we apply the methods of RRR, CDEP and ClArC for model
correction, and (4) (re-)evaluate the model's performance and remaining
sensitivity towards the artifact. Using two medical benchmark datasets for
Melanoma detection and bone age estimation, we apply our R2R framework to VGG,
ResNet and EfficientNet architectures and thereby reveal and correct real
dataset-intrinsic artifacts, as well as synthetic variants in a controlled
setting. Completing the XAI life cycle, we demonstrate multiple R2R iterations
to mitigate different biases. Code is available on
https://github.com/maxdreyer/Reveal2Revise
Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Test
The Model Parameter Randomisation Test (MPRT) is widely acknowledged in the
eXplainable Artificial Intelligence (XAI) community for its well-motivated
evaluative principle: that the explanation function should be sensitive to
changes in the parameters of the model function. However, recent works have
identified several methodological caveats for the empirical interpretation of
MPRT. To address these caveats, we introduce two adaptations to the original
MPRT -- Smooth MPRT and Efficient MPRT, where the former minimises the impact
that noise has on the evaluation results through sampling and the latter
circumvents the need for biased similarity measurements by re-interpreting the
test through the explanation's rise in complexity, after full parameter
randomisation. Our experimental results demonstrate that these proposed
variants lead to improved metric reliability, thus enabling a more trustworthy
application of XAI methods.Comment: 19 pages, 12 figures, NeurIPS XAIA 202
Explainable AI for time series via Virtual Inspection Layers
The field of eXplainable Artificial Intelligence (XAI) has witnessed significant advancements in recent years. However, the majority of progress has been concentrated in the domains of computer vision and natural language processing. For time series data, where the input itself is often not interpretable, dedicated XAI research is scarce. In this work, we put forward a virtual inspection layer for transforming the time series to an interpretable representation and allows to propagate relevance attributions to this representation via local XAI methods. In this way, we extend the applicability of XAI methods to domains (e.g. speech) where the input is only interpretable after a transformation. In this work, we focus on the Fourier Transform which, is prominently applied in the preprocessing of time series, with Layer-wise Relevance Propagation (LRP) and refer to our method as DFT-LRP. We demonstrate the usefulness of DFT-LRP in various time series classification settings like audio and medical data. We showcase how DFT-LRP reveals differences in the classification strategies of models trained in different domains (e.g., time vs. frequency domain) or helps to discover how models act on spurious correlations in the data
Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanations
Applying traditional post-hoc attribution methods to segmentation or object
detection predictors offers only limited insights, as the obtained feature
attribution maps at input level typically resemble the models' predicted
segmentation mask or bounding box. In this work, we address the need for more
informative explanations for these predictors by proposing the post-hoc
eXplainable Artificial Intelligence method L-CRP to generate explanations that
automatically identify and visualize relevant concepts learned, recognized and
used by the model during inference as well as precisely locate them in input
space. Our method therefore goes beyond singular input-level attribution maps
and, as an approach based on the recently published Concept Relevance
Propagation technique, is efficiently applicable to state-of-the-art black-box
architectures in segmentation and object detection, such as DeepLabV3+ and
YOLOv6, among others. We verify the faithfulness of our proposed technique by
quantitatively comparing different concept attribution methods, and discuss the
effect on explanation complexity on popular datasets such as CityScapes, Pascal
VOC and MS COCO 2017. The ability to precisely locate and communicate concepts
is used to reveal and verify the use of background features, thereby
highlighting possible biases of the model
Optimizing Explanations by Network Canonization and Hyperparameter Search
Explainable AI (XAI) is slowly becoming a key component for many AI
applications. Rule-based and modified backpropagation XAI approaches however
often face challenges when being applied to modern model architectures
including innovative layer building blocks, which is caused by two reasons.
Firstly, the high flexibility of rule-based XAI methods leads to numerous
potential parameterizations. Secondly, many XAI methods break the
implementation-invariance axiom because they struggle with certain model
components, e.g., BatchNorm layers. The latter can be addressed with model
canonization, which is the process of re-structuring the model to disregard
problematic components without changing the underlying function. While model
canonization is straightforward for simple architectures (e.g., VGG, ResNet),
it can be challenging for more complex and highly interconnected models (e.g.,
DenseNet). Moreover, there is only little quantifiable evidence that model
canonization is beneficial for XAI. In this work, we propose canonizations for
currently relevant model blocks applicable to popular deep neural network
architectures,including VGG, ResNet, EfficientNet, DenseNets, as well as
Relation Networks. We further suggest a XAI evaluation framework with which we
quantify and compare the effect sof model canonization for various XAI methods
in image classification tasks on the Pascal-VOC and ILSVRC2017 datasets, as
well as for Visual Question Answering using CLEVR-XAI. Moreover, addressing the
former issue outlined above, we demonstrate how our evaluation framework can be
applied to perform hyperparameter search for XAI methods to optimize the
quality of explanations
Explaining nonlinear classification decisions with deep Taylor decomposition
Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems such as image recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transparency, limiting the interpretability of the solution and thus the scope of application in practice. Especially DNNs act as black boxes due to their multilayer nonlinear structure. In this paper we introduce a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements. Although our focus is on image classification, the method is applicable to a broad set of input data, learning tasks and network architectures. Our method called deep Taylor decomposition efficiently utilizes the structure of the network by backpropagating the explanations from the output to the input layer. We evaluate the proposed method empirically on the MNIST and ILSVRC data sets